With abundance of easy-to-use Machine Learning Libraries, it is often appealing to apply them and achieve greater than 80% prediction accuracy in most cases. But, 'WHAT TO TRY NEXT?' is a question that buzz me and may be other aspiring Data Scientists too. During my course'Machine Learning -- Stanford Online' at Coursera, Prof. Andrew Ng helped me sail through it. I hope this article, which briefs his explanation during one of his lectures, will help many of us to understand the importance of'debugging or diagnosing a learning algorithm'. To start with, let's call out all the possibilities or'WHAT TO TRY NEXT?' when a hypothesis makes unacceptably large errors in its predictions or when there is a need to improve our hypothesis: We will revisit this table to make smart choices and create our TOOL BOX.